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ABSTRACT 

The Dutch Educational Opportunity Program was begun 
in 1988 to improve educational achievement of Dutch low socioeconomic 
status (SES) children and children from ethnic minorities* A 
longitudinal study was done of children in grades 4, 6, and 8 at 700 
primary schools. Every 2 years the children were tested for 
intelligence, arithmetic, and Dutch language. After evaluation of two 
testing periods, it appears that achievement among the disadvantaged 
groups has not improved in comparison with a representative reference 
group. However, when analyzing the data in more detail, taking into 
account the children's SES, ethnic background, and intelligence, 
remarkable differences can be found in effectiveness between schools 
in arithmetic and Dutch language proficiency. Data were gathered on 
about 50,000 students tested in grade 4, 6, and 8 in 1988 and in 1990 
in about 600 primary schools. Results suggest that in the future the 
quality of education should be monitored closely. Many of the 
failures in helping disadvantaged children result from the 
misconception that education should be adapted to the development 
level of the child. Education should be modeled around the concept 
that all students can reach desired goals, but some may require more 
time and instruction. (JPT) 
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Differences in school and instruction characteristics between highly, 
average and low effective schools. 



Greetje van der Werf 

RION, Institute for Educational Research 

University of Groningen, The Netherlands 

1. INTRODUCTION 

In 1988 we started the evaluation of the Dutch Educational Priority Program, that is 
aimed at the improvement of educational achievements of Dutch low SES children 
and children from ethnic minorities. The evaluation consists of longitudinal cohort 
studies of all children in grade 4, 6 and 8 of 700 primary schools. Every two years 
these children are tested for intelligence, arithmetic and Dutch language. Information 
about school and instruction characteristic the children are confronted with is 
collected from school principals and teachers. Until now three measurements have 
taken place, the data of two measurements have been analyzed. 
Overall, it appears that, since the introduction of the Priority Program, the 
achievements of the disadvantaged groups of children have not improved at all in 
comparison with the achievements of a reference group of children that is 
representive for all Dutch primary school children. So, in general the Priority Program 
did not appeal succesfull thus far. 

However, when analyzing the data of the schools more in detail, taking into account 
the childrens' SES, ethnic background and intelligence, we must conclude that there 
are remarkable differences in effectiveness between schools in terms of arithmetic and 
Dutch language achievement. The data of the first as well as the second measurement 
show that between 9 and 12% of the variance in students' achievements is 'between- 
school-variance', depending on grade and subject. However, only a few school and 
instruction characteristics can explain this variance. Besides those characteristics differ 
also with grade and subject. 

From other research it is well-known that the consistency of effectiveness across 
grades and school years is not very high (Van Batenburg, 1990; Bosker, 1991). This 
could explain why also the consistency of the related school and instruction 
characteristics is low. 

The aim of this paper is to look more in detail to the effectiveness of schools across 
grades and school years and to find out which variables on school and teacher level 
can explain the difference in consistent effectiveness between schools. 
The research questions are the following: 

1. Are there any schools that are highly, average or low effective across grades 
and across school years? 

2. To what extent do these groups of consistently high, average and low effective 
schools differ in achievement scores for different groups of students? 

3. Which school and instruction characteristics are related to the differences in 
consistent effectiveness? 



2. THEORETICAL FRAMEWORK. 



The general theoretical model of the research is given by the literature about school 
and instructional effectiveness. In the model school level variables are considered to 
be conditional for effective instruction within and across classrooms (Scheerens, 1989; 
Creemers, 1991). The most important school level variables are: educational 
leadership, emphasis on learning basic skills, high expectations of students' 
achievements and participation of parents within the school. As effective instructional 
variables are considered those that have appeared effective for students in general 
(Good & Brophy, 1986; Walberg, 1986; Creemers, 1991) as well as for specific groups 
of disadvantaged students (Slavin & Madden, 1989; Levine & Lezotte, 1990; Levin, 
1990). The following instruction variables are taken into account: opportunity to learn, 
time on task, grouping of students, evaluation and feedback and classroom climat. 

3. METHOD. 

3.1 Dataset 

In order to answer the research questions the data of all schools who co-operated 
with the two measurements of the evaluation of the priority program were analyzed. 
The dataset contains the data of about 50.000 pupils tested in grade 4, 6 an 8 in 1988 
and in 199o in about 600 primary schools, as well as the data about the instructional 
characteristics of teachers who were teaching the respective grades in 1988 and 1990, 
as well as the data about the school characteristics, gathered from the principals in 
1988 and 1990. 

3.2 Variables and instruments 

To determine the differences in consistent effectiveness between schools we used the 
students' arithmetic scores, because arithmetic scores in general are explained the 
most by schools and the least by individual student characteristics. The arithmetic 
scores were measured by an arithmetic test, containing 40 items. As co-variates were 
taken into account the students' scores on the performal part of an intelligence test 
and the students' weight factor. This weight factor is the indicator of the socio- 
economical and ethnic background of the children. The weight factor for a Dutch low 
SES child is 1.25, the weight factor for a low SES ethnic minority child is 1.90. The 
school and instruction variables to which eventual differences between schools can be 
ascribed were measured by written questionnaires, consisting of Likert scales or single 
items, to be filled in by principals and teachers. The reliability of the scales all were 
satisfying (.60 - .87), 

3.3 Procedure 

In order to answer the first research question firstly the data were analyzed by means 
of the multi level program VARCL (Longford, 1988) for each grade and each school 
year seperatly. As co-variates the students' intelligence score and their weight factor 
were taken into account. After that we computed for each school the intercept 



deviances from the grand mean for each grade and each school year seperatly. 
Schools with an intercept deviance of two or more standard deviations above or 
beneath the grand mean were selected as highly respectively low effective for the 
respective grades and school. years. Schools with an intercept deviance near zero were 
selected as average. Finally schools that were highly, average or *ow effective for all 
grades in both school years were selected as consistantly high, average or low 
effective. The differences in school and instruction variables between the three 
selected groups of schools were analyzed by means of one-way variance analyses with 
a significance level of p < .01. 



4.1 



RESULTS 



Differences in effectiveness between schools 



To determine the differences in effectiveness between schools for the different grades 
and school years separately, we used, as was said before, the multi level program 
VARCL. Firstly we specified the 'empty' model, the model without students' and 
school and instruction characteristics. Subsequently intelligence and weight factor were 
added as fixed co-variates to the model. The percentage of variance left indicates the 
extent to which schools differ in effectiveness. In table 1 the proportion of variance on 
pupil and school level are presented for arithmetic in grade 4, 6 and 8 in 1988 and 
1990. 



Table 1 Percentages of variance on pupil and school level per grade and per 
schoolyear (model with pupil co-variates). 



School year 1988 



Schoolyear 1 990 





grade 4 


grade 6 


grade 8 


grade 4 


grade 6 


grade 8 


Pupil level 


59% 


62% 


55% 


58% 


63% 


56% 


School level 


15% 


8% 


11% 


15% 


9% 


14% 


total 


74% 


70% 


66% 


73% 


72% 


70% 


var. expl. 


26% 


30% 


34% 


27% 


28% 


30% 
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In both schoolyears and in all grades a substantial proportion 'between-school- 
variance' is left. So it must be possible to select schools on differences in effectiveness. 
Therefore we rankordered the schools per grade per schoolyear on the intercepts 
(mean artithmetic scores after correction for individual weight factors and intelligence 
scores. Schools that deviate two or more standard deviations above or beneath the 
grand mean for each grade in each schoolyear were selected as consistently high or 
low effective. Schools with a deviance for each grade in each schoolyear near zero 
were selected as consistently average effective. From the 600 schools only 10 
appeared as consistently high effective, 12 as consistently low effective and 18 as 

5 



consistently average effective. 

In table 2 we can see what these schools mean for the arithmetic scores of the 
different groups of children. For the ease of survey only the arithmetic scores in grade 
8 in 1990 are presented. The scores are raw scores, i.e. they are not corrected for 
intelligence. 



Table 2 Mean arithmetic scores in grade 8, 1990, in highly, average and low 


effective schools per weight factor. 






weight factor 


mean score 


Low effective 


1.00 


22 




1.25 


18 




1.90 


16 


Average effective 


1.00 


28 




1.25 


25 




1.90 


24 


Highly effective 


1.00 


34 




1.25 


32 




1.90 


31 



Mean score reference group: 27 (100 : 29; 1.25 : 26; 1.90 : 23). 



On the highly effective schools all groups of children (also the 1.25 and 1.90 children, 
the target groups of the priority policy) score far above the national reference group. 
The average effective schools score for the 1.25 children a bit lower than the 1.25 
children in the reference groups, for the 1.90 children a bit higher compared to the 
1.90 children in the reference group. The low effective schools score for all groups o f 
children a beneath the national reference group. Furthermore it appears that the 
difference between the children from the target groups and the 1.00 children on the 
highly effective schools is smaller compared to the difference in the reference group. 
So a highly effective school not only is favourable for Dutch middle class children but 
also for Dutch low SES and ethnic minority children. 

To keep the comparison completely fair I also will show the estimates of the 
arithmetic scores for any other 1.00, 1.25 and 1.90 students, with comparable 
intelligence scores, for each of the three groups of schools. The overall mean score of 
all 1.00 students is 29. The estimated score for the 1.00 students on the three groups 
of schools is the sum of the overall mean score and the intercept deviance, tn the 
group of low effective schools the intercept deviance varies between -4 and -6. The 
estimated scores of 1.00 children on these schools are 4 till 6 points beneath 29. For 
the highly effective schools the intercept deviance varies between +4 and +6. The 
estimated scores of the 1.00 children on these schools are 4 till 6 points above 29. The 
same can be done for the 1.25 and 1.90 children. The estimated scores for the 1.25 
children on highly and low effective schools are 4 till 6 points above respectively 
beneath 26, so between 32 and 20. For the 1.90 children the estimates are 4-6 points 
above or beneath 23, so between 29 and 17. On a test with 40 items these are 
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remarkable differences. 



Figure 1 Estimated scores for highly, average and low effective schools 
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Between the groups of schools differences are tested in school and instruction 
characteristics, to see wether the differences in effectiveness have something to do 
with the educational organization and content of the schools. 
The results are in table 3. 



Table 3 Differences in school and instruction characteristics between schools 



Characteristics 


Mean scores 




Range of 










score 




low eff. 


average 


hiahlv 






eff. 


eff. 


eff. 














Educational leadership 


1ft 

I yJ 


19 


13 


11-33 


Pnnconci ic cahm it nn^lQ 

wUI loci loUo uUUU L tjUulo 


6 3 


7 3 


6 8 


4-12 


ln<?tn irtionai nharantpri^tin^ 

11 1 0 LI Li w L t W 1 ICll wl lal OvlUl lOLIwO 










whole classroom ir.ctr. 


3.2 


3.7 


4.2 


1-5 


ability grouping 


2.7 


2.0 


1.8 


1-5 


minimum goals 


2.1 


1.5 


1.4 


1-3 


% time for organization 


4.6 


3.5 


1.9 




% time for evaluating learning tasks 


5.4 


6.8 


9.8 




% time for classroom instr. 


9.4 


11.7 


15.7 




% time seatwork 


3.4 


2.6 


2.0 




amount of lessons spent on arith. 


4.5 


5.0 


5.0 




correcting and hearing learning tasks 


3.0 


2.9 


2.3 


1-4 


homework 


5.3 


6.0 


7.4 


3-15 


use of method 


2.2 


1.7 


1.7 


1-3 


% learning content 


77 


96 


95 




registration mastering learning content 3.3 


3.8 


4.1 


1-5 



The groups of schools appear to differ on only two school level characteristics: 
educational leadership and consensus about the educational goals. Striking is that both 
characteristics are more present on the low and average effective schools. With regard 
to the instructional characteristics the picture is clear and consistent. The high 
effective schools give more instruction towards whole class rooms, work less in ability 
groups, have more often minimum goals, spend more hours to arithmetic, spend less 
time of the lesson on organizational activities and more time on classroom instruction 
and evaluation of learning tasks. On the other hand teachers give less individual 
seatwork and correct and hear less often learning tasks. This could be related to the 
fact that they evaluate the work with the whole classroom. Highly effective schools 
give more homework and more often additional homework to disadvantaged students. 
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Concerning the methods, they use this more often in agreement with the prescriptions 
and they treat more learning content during the school year. Finally they registrate 
more often whether the students master the learning content. 



5. CONCLUSION 

For the future of the educational priority program it is very important that the quality 
of education will be watched carefully. As a consequence of the growing attention for 
specific measures for disadvantaged children, like for example pre school projects, 
second language education, etc, the importance of effective education within 
classrooms is doomed to be pushed to the background. This leads to a situation in 
which the eventual advantage of specific measures could be lost because 
disadvantaged children are confronted with contra productive educational strategies 
during a substantial amount of their school period. A lot of these contra productive 
strategies originated, in my opinion, from the deeply rooted mis conception that 
education should be adapted to the developmental level of individual students. The 
starting point however should be that all students can reach the desired final 
educational goals, but that some students therefore need more time, more instruction 
and more excersize. Besides, also the quality of instruction is important: clear 
instruction, structuring the learning content, evaluating adequatly the mastery of the 
minimum goals and giving positive feedback. 

So, measures to improve the quality of education within classrooms, are in my 
opinion, the most important to improve the educational opportunities of Dutch low 
SES and ethnic minority children. 
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